Incremental Algorithm Research Articles

Planning, Design, and Execution of a Quantum Incremental Clustering Algorithm System for the Examination of Unsupervised Data

The exponential growth of data in various domains has underscored the need for efficient unsupervised learning algorithms to discover hidden patterns and structures within large datasets. Traditional clustering algorithms often face challenges in handling dynamic datasets and require periodic retraining, making them less suitable for real-time applications. In response to these challenges, this research introduces a Quantum Incremental Clustering Algorithm System (QICAS) designed to adapt to evolving datasets and provide timely insights into the underlying structures. The proposed QICAS is based on quantum computing principles, leveraging the unique properties of qubits and quantum superposition to perform clustering tasks more efficiently than classical counterparts. The research focuses on the planning, design, and execution of the QICAS, ensuring a comprehensive understanding of the algorithm's capabilities and performance.

Iterative Train Scheduling under Disruption with Maximum Satisfiability

This paper proposes an iterative Maximum Satisfiability (MaxSAT) approach designed to solve train scheduling optimization problems. The generation of railway timetables is known to be intractable for a single track. We consider hundreds of trains on interconnected multi-track railway networks with complex connections between trains. Furthermore, the proposed algorithm is incremental to reduce the impact of time discretization. The performance of our approach is evaluated with the real-world Swiss Federal Railway (SBB) Crowd Sourcing Challenge benchmark and Periodic Event Scheduling Problems benchmark (PESPLib). The execution time of the proposed approach is shown to be, on average, twice as fast as the best existing solution for the SBB instances. In addition, we achieve a significant improvement over SAT-based solutions for solving the PESPLib instances. We also analyzed real schedule data from Switzerland and the Netherlands to create a disruption generator based on probability distributions. The novel incremental algorithm allows solving the train scheduling problem under disruptions with better performance than traditional algorithms.

Universal approximation property of stochastic configuration networks for time series

For the purpose of processing sequential data, such as time series, and addressing the challenge of manually tuning the architecture of traditional recurrent neural networks (RNNs), this paper introduces a novel approach-the Recurrent Stochastic Configuration Network (RSCN). This network is constructed based on the random incremental algorithm of stochastic configuration networks. Leveraging the foundational structure of recurrent neural networks, our learning model commences with a modest-scale recurrent neural network featuring a single hidden layer and a solitary hidden node. Subsequently, the node parameters of the hidden layer undergo incremental augmentation through a random configuration process, with corresponding weights assigned structurally. This iterative expansion continues until the network satisfies predefined termination criteria. Noteworthy is the adaptability of this algorithm to handle time series data, exhibiting superior performance compared to traditional recurrent neural networks with similar architectures. The experimental results presented in this paper underscore the efficacy of the proposed RSCN for sequence data processing, showcasing its advantages over conventional recurrent neural networks in the context of the performed experiments.

Penalized Bayesian forward continuation ratio model with application to high-dimensional data with discrete survival outcomes.

While time-to-event data are often continuous, there are several instances where discrete survival data, which are inherently ordinal, may be available or are more appropriate or useful. Several discrete survival models exist, but the forward continuation ratio model with a complementary log-log link has a survival interpretation and is closely related to the Cox proportional hazards model, despite being an ordinal model. This model has previously been implemented in the high-dimensional setting using the ordinal generalized monotone incremental forward stagewise algorithm. Here, we propose a Bayesian penalized forward continuation ratio model with a complementary log-log link and explore different priors to perform variable selection and regularization. Through simulations, we show that our Bayesian model outperformed the existing frequentist method in terms of variable selection performance, and that a 10% prior inclusion probability performed better than 1% or 50%. We also illustrate our model on a publicly available acute myeloid leukemia dataset to identify genomic features associated with discrete survival. We identified nine features that map to ten unique genes, five of which have been previously associated with leukemia in the literature. In conclusion, our proposed Bayesian model is flexible, allows simultaneous variable selection and uncertainty quantification, and performed well in simulation studies and application to real data.

Cross-Class Feature Augmentation for Class Incremental Learning

We propose a novel class incremental learning approach, which incorporates a feature augmentation technique motivated by adversarial attacks. We employ a classifier learned in the past to complement training examples of previous tasks. The proposed approach has an unique perspective to utilize the previous knowledge in class incremental learning since it augments features of arbitrary target classes using examples in other classes via adversarial attacks on a previously learned classifier. By allowing the Cross-Class Feature Augmentations (CCFA), each class in the old tasks conveniently populates samples in the feature space, which alleviates the collapse of the decision boundaries caused by sample deficiency for the previous tasks, especially when the number of stored exemplars is small. This idea can be easily incorporated into existing class incremental learning algorithms without any architecture modification. Extensive experiments on the standard benchmarks show that our method consistently outperforms existing class incremental learning methods by significant margins in various scenarios, especially under an environment with an extremely limited memory budget.

Clustering-based incremental learning for imbalanced data classification

Imbalanced data classification presents a significant challenge when there is a substantial disparity in sample sizes across different classes. This issue severely affects classifier accuracy in predicting minority classes, hampering numerous real-world applications. Traditional methods address data imbalance by using undersampling or oversampling techniques. However, these methods may lead to information loss during sample reduction or introduce noise and model bias through synthetic sample generation. In this paper, we introduce DRIL, an innovative clustering-based incremental learning approach designed to overcome these limitations and improve the classification of minority class samples. Specifically, we employ a “two-step clustering” method to rebalance the dataset, partitioning it into similar and representative sub-dataset. Subsequently, incremental learning is applied to enable the classifier to gradually acquire knowledge about these sub-data, establishing a comprehensive understanding of all features present in the imbalanced dataset. Experimental results on twenty datasets demonstrate that our incremental learning-based algorithm outperforms baseline methods in correctly classifying minority classes while exhibiting improved precision and F1 score performance.

Survey of continuous deep learning methods and techniques used for incremental learning

Neural networks and deep learning algorithms are designed to function similarly to biological synaptic structures. However, classical deep learning algorithms fail to fully capture the need for continuous learning; this has led to the advent of incremental learning. Incremental learning adds new challenges that are handled differently by modern state-of-the-art approaches. Some of these include: utilization of network memory as additional knowledge increases the size of the network, open-set recognition to be able to identify unrecognized information, and efficient knowledge distillation as most incremental learning algorithms are prone to catastrophic forgetting of previously learned knowledge. Recent advancements achieve incremental learning through a multitude of methods. Most methods are characterized by augmenting the normal algorithm of neural network training by both directly modifying the neural network structure and by adding additional learning steps. This paper analyzes and provides a comprehensive survey of existing methods and various techniques used for incremental learning. A novel categorization of the methods is also introduced based on recent trends of the state-of-the-art solutions. The study focuses on methods that provide incremental learning success as well as discusses emerging patterns in new research.

Adaptive regularized ensemble for evolving data stream classification

Extracting knowledge from data streams requires fast incremental algorithms that are able to handle unlimited processing and ever-changing data with finite memory. A strategy for this challenge is the use of ensembles owing to their ability to tackle concept drift and achieve highly accurate predictions. However, ensembles often require a lot of computational resources. In this study, we propose a novel ensemble-based classification algorithm for data streams, Adaptive Regularized Ensemble (ARE), with low demand for computational resources. The algorithm combines strategies that contribute to high prediction accuracy using only incorrectly classified instances into the training step, random-sized feature subspace for each ensemble element and classifier selection for final ensemble voting. After an extensive experimental study, ARE exhibited high predictive performance and outperformed state-of-the-art ensembles on data streams for real and synthetic datasets while requiring a low processing time and memory consumption.

Linking Entities across Relations and Graphs

This article proposes a notion of parametric simulation to link entities across a relational database 𝒟 and a graph G . Taking functions and thresholds for measuring vertex closeness, path associations, and important properties as parameters, parametric simulation identifies tuples t in 𝒟 and vertices v in G that refer to the same real-world entity, based on both topological and semantic matching. We develop machine learning methods to learn the parameter functions and thresholds. We show that parametric simulation is in quadratic-time by providing such an algorithm. Moreover, we develop an incremental algorithm for parametric simulation; we show that the incremental algorithm is bounded relative to its batch counterpart, i.e., it incurs the minimum cost for incrementalizing the batch algorithm. Putting these together, we develop HER , a parallel system to check whether ( t, v ) makes a match, find all vertex matches of t in G , and compute all matches across 𝒟 and G , all in quadratic-time; moreover, HER supports incremental computation of these in response to updates to 𝒟 and G . Using real-life and synthetic data, we empirically verify that HER is accurate with F-measure of 0.94 on average, and is able to scale with database 𝒟 and graph G for both batch and incremental computations.

Unit vector template controlled grid integrated and solar fed BLDC drive-based water pumping system

With increased population and energy demand, the need to utilize renewable energy becomes so vital. The need to manage food demand globally also emphasizes the support to agriculture. This article proposes a unit vector template-controlled grid-integrated solar Photovoltaic (PV) powered three-phase Brushless Direct Current (BLDC) drive-based water pumping system. The proposed system includes solar photovoltaic, boost converter, voltage source inverter, single phase grid supply, single phase bidirectional voltage source converter, and BLDC-driven water pump. The solar PV is connected to the dc grid through the boost converter. An incremental conductance algorithm performs the Maximum Power Point Tracking (MPPT) of a Photovoltaic array. The power exchange between the dc-bus and the utility grid has been achieved through a unit vector template algorithm-controlled bidirectional voltage source converter. It allows a consumer to operate a water pump at maximum capacity for the whole day regardless of the weather and to supply a single-phase power grid when water pumping is unnecessary. A 3-phase BLDC motor-based water pump is driven by a 3-phase voltage source inverter, which is used for electronic commutation and to regulate the motor speed. The performance of the proposed system has been evaluated for different modes of peak, rms voltage and motor speed has been maintained around 255 V and 180 V and 2500 rpm respectively in all the modes of operation. Its practicality and trustworthiness are shown by different simulation results using MATLAB/Simulink version 2022bsoftware.

A Procedural Constructive Learning Mechanism with Deep Reinforcement Learning for Cognitive Agents

Recent advancements in AI and deep learning have created a growing demand for artificial agents capable of performing tasks within increasingly complex environments. To address the challenges associated with continuous learning constraints and knowledge capacity in this context, cognitive architectures inspired by human cognition have gained significance. This study contributes to existing research by introducing a cognitive-attentional system employing a constructive neural network-based learning approach for continuous acquisition of procedural knowledge. We replace an incremental tabular Reinforcement Learning algorithm with a constructive neural network deep reinforcement learning mechanism for continuous sensorimotor knowledge acquisition, thereby enhancing the overall learning capacity. The primary emphasis of this modification centers on optimizing memory utilization and reducing training time. Our study presents a learning strategy that amalgamates deep reinforcement learning with procedural learning, mirroring the incremental learning process observed in human sensorimotor development. This approach is embedded within the CONAIM cognitive-attentional architecture, leveraging the cognitive tools of CST. The proposed learning mechanism allows the model to dynamically create and modify elements in its procedural memory, facilitating the reuse of previously acquired functions and procedures. Additionally, it equips the model with the capability to combine learned elements to effectively adapt to complex scenarios. A constructive neural network was employed, initiating with an initial hidden layer comprising one neuron. However, it possesses the capacity to adapt its internal architecture in response to its performance in procedural and sensorimotor learning tasks, inserting new hidden layers or neurons. Experimentation conducted through simulations involving a humanoid robot demonstrates the successful resolution of tasks that were previously unsolved through incremental knowledge acquisition. Throughout the training phase, the constructive agent achieved a minimum of 40% greater rewards and executed 8% more actions when compared to other agents. In the subsequent testing phase, the constructive agent exhibited a 15% increase in the number of actions performed in contrast to its counterparts.

Distributed Feature Selection for Power System Dynamic Security Region Based on Grid-Partition and Fuzzy-Rough Sets

In order to satisfy the requirements of modern online security assessment of power systems with continuously increasing complexity in terms of structure and scale, it is desirable to develop a power system dynamic security region (DSR) analysis. However, data-driven methods suffer from expensive model training costs and overfitting when determining DSR boundaries with high-dimensional grid features. Given this problem, a distributed feature selection method based on grid partition and fuzzy-rough sets is proposed in this paper. The method first employs the Louvain algorithm to partition the power grid and divide the original feature set so that high-dimensional features can be allocated to multiple computational units for distributed screening. At this point, the connections between features of different computational units are minimized to a relatively low level, thereby avoiding large errors in the distributed results. Then, an incremental search algorithm based on the fuzzy-rough set theory (FRST) is used for feature selection at each computational unit, which can effectively take into account the intrinsic connections between features. Finally, the results of all computational units are integrated in the coordination unit to complete the overall feature selection. The experimental results based on the IEEE-39 bus system show that the proposed method can help simplify the power system DSR analysis with high-dimensional features by screening the critical features. And compared with other commonly used filter methods, it has higher screening accuracy and lower time costs.

An incremental clustering algorithm based on semantic concepts

An incremental clustering algorithm based on semantic concepts

Adaptive Hybrid NFS-MSOGIQ Control Technique for Improving Power Quality in Solar PV Distributed Generation System

A two-stage circuit configuration with 3-phase utility grid assisted solar power generation system is designed. In order to track the solar PV arrays maximum peak power (MPP), a DC boost converter with Composite Incremental Conductance (InC) MPP technique is employed in first stage. In Second stage, a grid-tied four-leg DC-AC converter, is employed to provide the solar power that was extracted into the utility grid and to enhancing power quality. An adaptive hybrid Multi-Second Order Generalized Integrator-Quadrature (MSOGI-Q) control algorithm, in conjunction with Neuro Fuzzy system (NFS) is proposed for controlling this DC-AC converter. MSOGI-Q reference current generation strategy is designed to mitigate current harmonics by retrieving the fundamental constituents from non-linear load. The Neuro Fuzzy system along with PID controller, by adjusting the voltage from dc link keeps consistent power in between AC and DC sides for varying isolation condition. To get the peak power out of a PV panel even in different environmental conditions, the composite incremental conductance algorithm is used. To upgrade the dynamic performance of voltage variation at Point of Common Coupling (PCC) a feedforward descriptor for Photovoltaic contributions has been implemented. In addition, a comparison between the proposed NFS-MSOGIQ control technique and the existing adaptive control technique is done to assess the effectiveness of the proposed adaptive control technique. The proposed adaptive control technique for the distributed generation system is designed and stimulated in MATLAB/SIMULINK and the simulation outcomes verify the effectiveness of the proposed control technique under various transition circumstances. A laboratory prototype model of the proposed system is developed for theoretical analysis and its results are found to be compliant with IEEE standards. Furthermore, THD values of voltage and current are found to be within standard limits.

Optimal control of microgrid using ALO optimization MPPT with power quality enhancement

AbstractThe paper presents a normalized, varying step size-based least mean square-based control for a standalone microgrid and Ant Lion optimization-based maximum power point tracking. This modified version of the incremental conductance algorithm addresses issues like slow dynamic response, fixed step size issues, and steady state oscillations. Comparative analysis with well-known techniques shows that Ant Lion optimizes the tracking of maximum power points more accurately, with fewer oscillations and increased efficiency. Power management and power quality are maintained through proposed adaptive voltage source converter control. The proposed voltage control shows better performance under various dynamic and steady conditions.

Graph-based few-shot incremental learning algorithm for unknown class detection

Few-shot learning, a promising technique for acquiring new concepts from limited data, assumes that testing samples belong to “unknown classes” and are regarded as new knowledge. However, real-world scenarios introduce uncertainty about the class membership of testing samples. To address this uncertainty, we propose a novel challenge of few-shot incremental unknown class detection, aligning more closely with practical situations. Open set recognition can classify known class samples and reject unknown class samples to mitigate the uncertainty, but it struggles to address the critical limitation of having few available samples. To tackle both uncertainty and limitation, we propose a graph-based few-shot incremental learning algorithm for unknown class detection, which includes four components. First, a feature extractor learns from the base dataset during training and is subsequently fixed for embedding node features from the novel dataset during inference. Then, embedded node features, along with their corresponding prototypes, contribute to graph generation and edge construction. Third, a mixed-rejection strategy is proposed to determine the class membership of testing samples. Finally, a novel class is treated as a new known class, engaging the embedded node features in graph update and edge reconstruction. Evaluation on benchmark datasets with varying structures, including USTC-TFC2016 and miniImageNet datasets, demonstrates that our proposed algorithm outperforms classical open set recognition algorithm in few-shot incremental learning for unknown class detection, which offers promising performance and potential for practical applications in real-world scenarios.

A novel maximum power point tracking with hybrid control algorithm for automotive thermoelectric generator system.

To dynamically track the maximum power of an automotive thermoelectric generator (ATEG) system in real-time, this study introduces a novel maximum power point tracking (MPPT) algorithm that integrates Kalman filtering and fuzzy control. Employing a two-phase interleaved parallel DC-DC boost converter in the MPPT controller effectively reduces current ripple and switch loss. Results demonstrated a significant improvement in tracking time compared to the traditional incremental conductance algorithm, attributed to the elimination of high-frequency components in output power by the Kalman filter. The novel algorithm exhibits enhanced tracking stability through the application of fuzzy control. Ultimately, the tracking accuracy of the novel algorithm surpasses that of the incremental conductance algorithm by 5.2%, achieving an impressive 94.9%. This study, therefore, presents a valuable contribution to a novel MPPT algorithm for precisely and rapidly tracking the global maximum power points of the ATEG system throughout the entire vehicle driving cycle.

Incremental reinforcement learning and optimal output regulation under unmeasurable disturbances

In this paper, we propose novel data-driven optimal dynamic controller design frameworks, via both state-feedback and output-feedback, for solving optimal output regulation problems of linear discrete-time systems subject to unknown dynamics and unmeasurable disturbances using reinforcement learning (RL). Fundamentally different from existing research on optimal output regulation problems and RL, the proposed procedures can determine both the optimal control gain and the optimal dynamic compensator simultaneously instead of presetting a non-optimal dynamic compensator. Moreover, we present incremental dataset-based RL algorithms to learn the optimal dynamic controllers that do not require the measurements of the external disturbance and the exostate during learning, which is of great practical importance. Besides, we show that the proposed incremental dataset-based learning methods are more robust to a class of measurement noises with arbitrary magnitudes than routine RL algorithms. Comprehensive simulation results validate the efficacy of our methodologies.

BASS: Broad Network Based on Localized Stochastic Sensitivity.

The training of the standard broad learning system (BLS) concerns the optimization of its output weights via the minimization of both training mean square error (MSE) and a penalty term. However, it degrades the generalization capability and robustness of BLS when facing complex and noisy environments, especially when small perturbations or noise appear in input data. Therefore, this work proposes a broad network based on localized stochastic sensitivity (BASS) algorithm to tackle the issue of noise or input perturbations from a local perturbation perspective. The localized stochastic sensitivity (LSS) prompts an increase in the network's noise robustness by considering unseen samples located within a Q -neighborhood of training samples, which enhances the generalization capability of BASS with respect to noisy and perturbed data. Then, three incremental learning algorithms are derived to update BASS quickly when new samples arrive or the network is deemed to be expanded, without retraining the entire model. Due to the inherent superiorities of the LSS, extensive experimental results on 13 benchmark datasets show that BASS yields better accuracies on various regression and classification problems. For instance, BASS uses fewer parameters (12.6 million) to yield 1% higher Top-1 accuracy in comparison to AlexNet (60 million) on the large-scale ImageNet (ILSVRC2012) dataset.

Optimizing The Maximum Power of Photovoltaic System Using Modified Incremental Conductance Algorithm Operating Under Varying Dynamic Climatic Conditions

Optimizing The Maximum Power of Photovoltaic System Using Modified Incremental Conductance Algorithm Operating Under Varying Dynamic Climatic Conditions